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Identifiers-Racine, Wisconsin . 

The establishment of Research and Instructional CR and I) Units is discussed in 
terms of their utility and applicability for research needs. Thejse units are said to 
provide a propitious setting for rigorous educational research. Two experiments in R 
and I units are described-one dealing with ways to implement feedback to students 
on their school performance and another concerned with the effectiveness of older 
children as models for younger children of the same ethnic background. Results from 
these first experiments are being used to design further studies of teaching-learning 
theory. (NH) " 
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CONTROLLED EXPERIMENTATION 
IN 

RESEARCH AND INSTRUCTIONAL UNITS* 



Mary R. Quilling 

Laboratory of Experimental Design 
University of Wisconsin 



Educational researchers have long been plagued with the problem of 



finding enough experimental units to conduct research in the school 



setting. Typically, intact classrooms are available to the researcher, 



and the classroom is the only possible unit of randomization. In these 



instances the proper unit of analysis is the mean of each classroom. 



Of course, a large number of classrooms involving hundreds of children 



is required to provide data for a sensitive statistical test. 



In the event that pupils within classrooms may be randomly assigned 
to the experimental treatments, a different problem is encountered. 



Experimental arrangements, such as the process of randomization and 
splitting into groups, the use of different classrooms and strange 



teachers, may be so unusual to the pupils involved, that these arrange- 
ments interact with the treatment. Such reactive arrangements, as they 



are termed by Campbell and Stanley (1963), jeopardize the generaliza- 



bility of results. 

In response to these experimental design problems, Research and 
Instruction (R & I) Units were conceived and established last year in 



elementary schools of five Wisconsin cities. An R & I Unit is staffed 
by a Unit leader, 3-6 certified teachers, and several para-professionals, 



The Unit leader typically has some advanced training in curriculum and 



educational psychology, including methodology. This team is responsible 



Paper presented at the annual meeting of the American Educational 
Research Association, February 8-10, 1968, in Chicago, Illinois. 



*The research reported herein was conducted pursuant to Contract OE 
5-10-154 of the U.S. Office of Education. 
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for the education of 100 to 200 children, perhaps from two or three 
grade levels. While distinctive features of this organizational plan 
incorporate the better features of team teaching, let us concentrate 
on the research capability of these units. 

Of major importance is the flexibility which the Unit organization' 
permits. Teachers and pupils typically change rooms as pupils are re- 
grouped for instruction throughout the day. Assignment of pupils to 
new groups, instructors, or classrooms thus is an ordinary rather than 
unusual experience. Experimental arrangements requiring random assign 
ment of pupils to groups are thus far less likely to react with the 
treatment than would be the case in experiments involving pupils from 
self-contained classrooms. 

Teachers as well as pupils may be randomly assigned to treatments; 
furthermore teachers may be rotated among treatments so that a poten- 
tial source of confounding is eliminated. In the R & I Unit staff the 
experimenter has persons to administer the treatments who are both 
qualified to teach and have some appreciation of requirements of experi- 
mental rigor. Use of teachers as experimenters not only makes the 
research generalizable to usual school situations where children are 
taught by certified teachers; it also exposes the teacher to new methods, 
contributing to his professional growth. Thus we may conclude that the 
R & I Unit provides not only a propitious setting for the conduct of 
educational research, in which several shortcomings of other arrangements 
are overcome, it also has some beneficial effects as well. 

This is not to say that all the problems of conducting a true ex- 
periment in the school setting are solved. Strictly speaking, performing 
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the randomization on pupils is a necessary but not sufficient condition | 

j 

for their being treated as the unit of analysis. If the pupils are j 

! 

% 

instructed as a group after randomization, then intrasession history, ; 

* 

as well as the treatment, can affect the measurements gathered on a j 



particular group. In several R & I Units last year, however, the j 

jj 

experimental treatments were individually applied, and thus even this 

j 

requirement for a true experiment was met. 

I 

j 

Let us now consider two of the many experiments which were conduc- i 

j 

ted last year in R & I Units. The experimental treatments in each j 



instance were individually applied after pupils were randomly assigned 
to treatments. Both experiments represent attempts to translate moti- 
vational theory into practice in the school setting. The significance 
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level for testing each null hypothesis was set prior to the experiment \ 



at .10, a level considered appropriate for preliminary research. 

The first experiment was concerned with effective ways of implement- 
ing the principle that pupils should receive informative feedback rela- 
tive to their school performance. Conducted in an inner-city school 
in Racine, Wisconsin, the experiment was designed to assess the effect 
of feedback on acquisition of mathematics concepts. The subjects were 
72 children in a second grade R & I Unit classified as "disadvantaged" 
under Title I terms. The IQ's on the Kuhlman-Anderson ranged from 
72-136, with a mean of 101. In October of second grade the mean grade 
equivalent score on the arithmetic subtest of the Stanford Achievement 
Test, Primary I battery was 1.6. Pupils were stratified by age — younger 
or older — and by sex. Twenty-four children, six from each age-sex group 
were randomly selected as experimental subjects. The remainder of the 
group served as a control. 
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All pupils received individualized mathematics instruction. To 



assist teachers in assessing each individual’s progress, an individual 



record folder was designed. All the major concepts and subconcepts 
in mathematics were identified and compiled into a type of checklist 
in the individual folder. Beside each concept was a square to be 
colored in as an indication of the mastery of that concept. Diagnostic 
tests were developed for each main concept and subconcept. The format 
of the folder made it appropriate for use in parent-teacher and pupil- 



teacher conferences. 



After each child’s initial standing was determined using diagnos- 



tic tests and teacher observations, instruction was prescribed at an 



appropriate level. As many as four different instructional groups, each 
focusing on a different concept, were conducted simultaneously. Children 
were continually shifted among groups as concepts were mastered and new 
skills required. When a child appeared to have persistent difficulty 
in grasping one concept over a long period of time, he was allowed to 



go on to another group and return to the troublesome concept later. 



In this way, no child met consistent frustration, and individual mo- 



bility was assured. 



The experimental pupils additionally met individually with a 
teacher for five minutes each week to discuss progress and identify 



goals for the following week. At this time the child was allowed to 



color in the square for the concepts acquired during the week and was 
praised and encouraged by the teacher. Each teacher met individually 
with six of the experimental students every week for five weeks; then 



teachers were rotated. In this way, the effect of a particular teacher 



on a child's progress was minimized. The duration of the experiment 



1 






was 20 weeks. Each pupil, experimental or control, had access to his 
folder at all times. 

Two dependent variables were used to assess the effects of the 
experimental treatment— -the number of concepts mastered (or squares 
colored in) during the experiment, and the score obtained on a teacher- 
constructed posttest, which sampled the tasks enumerated in the mathe- 
matics folder. The means for each group are presented in Table 1. 

Table 1 

Means on Dependent Variables for 
Experimental and Control Groups 





Group 






Experimental 


Control 


Concepts mastered 


100 


75.0 


Score on posttest 


79 


63.5 



A multivariate analysis of variance was performed on the data. 
The results, presented in Table 2, indicate that the treatment was 
significant at the .10 level. 



Table 2 



Tests of Significance Using Wilks 1 Lambda Criterion 





df 


F 


Probability 


Treatment (T) 


2/5i 


2.479 


.094 


Sex (s) 


2/51 


1.540 


.224 


Age (A) 


2/51 


1.888 


.162 


T x S 


2/51 


.968 


.387 


T x A 


2/51 


.260 


.772 


S x A 


2/51 


5.842 


.005 


T x S x A 


2/51 


2.567 


.087 













Since the superior performance of the experimental group can be 
attributed to social interaction, goal setting behavior, and praise, 
as well as to informative feedback, experiments have been designed this 
year to clarify the contributions of specific components of the treat- 
ment. In summary, the preliminary experimentation identified a motiva- 
tional procedure which was successful in the classroom setting; thus 
encouraged, the teaching staff is ready to conduct more rigorous experi- 
ments which will further contribute to knowledge of the school applica- 
tion of theoretical constructs. Involvement of school personnel in 
generating the questions and executing the treatments has in this instance, 
made possible the iterative experimentation so necessary to learn why a 
procedure works. In an iterative cycle of experimentation, the results 
of an initial experiment are used to plan a subsequent, refined experi- 
ment in which further information may be gained. 

A second experiment was planned to learn whether older children are 
effective models for younger children of the same ethnic background. 
Subjects in this experiment were educationally disadvantaged children 
in a Milwaukee inner-city school. The majority of subjects were Negro, 
and handicapped by such characteristics as short attention span, poor 
self-concept, inadequate motor skills and low IQ. The Pintner- 
Cunningham test administered in the fall to the first graders revealed 
a mean IQ of 81 with range being 55-105. 

Fifty-seven children in the first grade R & I Unit were involved in 
the experiment. From these, 22 (11 boys, 11 girls) were randomly 
selected as experimental subjects to work with models. The remaining 
35 served as the control group. Similarly, 22 sixth— graders (11 boys, 11 
girls) were selected from the group of sixth-graders nominated by their 
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teachers to be appropriate models. The older pupils came from the 
same physical and socio-economic environment as the primary children. 

Each sixth-grade "helper" was randomly assigned to an experimental 
subject of the same sex. The older-younger pairs so formed spent one- 
half hour per week together playing arithmetic games, manipulating 
concrete objects and helping with problem solving in an Instructional 
Materials Center environment. In order to minimize the disruption of 
schedules, the models reported at times most convenient to their indi- 
vidual timetables. Additionally, the sixth-graders were briefed for 
half an hour each week on appropriate activities for the younger children 
and reinforcement procedures. The three primary teachers each took 
responsibility for supervising the experimental treatment on a rotating 

basis . 

A1 1 primary children received the same mathematics instruction four 
days a week. A televised mathematics program. Patterns in Ar ithmet ic , 
Grade 1 was used, together with an accompanying manual and teacher-made 
worksheets. Large and small group instruction was used as appropriate. 

In addition, individual instruction was given to pupils when considered 
necessary. 

A teacher-constructed test was designed to measure acquisition of 
the mathematics concepts presented during the fifteen-week experiment. 
The test was administered to all first graders at the conclusion of the 
experiment . 

The means of experimental boys and experimental girls were higher 
than those of control groups of each sex, as Table 3 indicates. 
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Table 3 

Means on Teacher-Constructed Test 







Boys 


Girls 




Mean 


Experimental 




16.00 


21.27 




18.63 


Control 




13.78 


17.84 




15.80 


The analysis of 


variance performed on the scores indicated that 


both treatment and sex were 


significant sources of 


variation 


. The 


statistical analysis 


is presented in Table 4. 










Table 


4 






Analysis of Variance on 


Scores 


of Teacher-Constructed Test 


Source 


df 




MS 


F 




Treatments (T) 


1 




108.0886 


3.3822 


p <.0716 


Sex (S) 


1 




295.9425 


9.2604 


p <.0037 


TS 


1 




5.0827 


.1590 




Error 


53 




31.9015 







Evaluation of the experiment also indicated that all experimental 

pairs were not compatible. Thus experiments in which the pairings are 

/ 

maximized for compatibility and model— effectiveness are a logical out- 
growth of the one reported here. • 

Experience to date indicates that given significant results, the 
^teaching staffs are eager to incorporate the treatment into the instruc- 
tional program or to investigate further the effectiveness of the 
procedure. The interplay of ideas of the classroom teacher and R & D 
specialist — be he an educational psychologist, a curriculum expert, or 
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research methodologist — have resulted in empirical evidence of the 



applicability in the school setting of principles derived from motiva- 
tional theory. That instruction has been improved simultaneously is 
evident from the field testing results* of schools with R & I Units. 



In summary, the R & I Unit is so organized that the conduct of 
true experiments in the school setting is possible. Staffing and phy- 
sical arrangements allow random assignment and individual treatment 
of pupils. Exemplars of such experiments, in which motivational theory 
was translated into practice, were given. Of promise for the future 
is the climate for iterative experimentation which has been created. 
Already results from initial experiments have been used to design sub- 
sequent experiments by which teaching-learning theory may be further 
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